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Abstract 

NASA is currently investigating a new concept of 
operations for the National Airspace System, designed to 
improve capacity while maintaining or improving current 
levels of safety. This concept, known as Distributed Air / 
Ground Traffic Management (DAG-TM), allows 
appropriately equipped “autonomous'’ aircraft to maneuver 
freely for flight optimization while resolving conflicts with 
other traffic and staying out of special use airspace and 
hazardous weather. While Airborne Separation Assurance 
System (ASAS) tools would normally allow pilots to 
resolve conflicts before they become hazardous, evaluation 
of system performance in sudden, near-term conflicts is 
needed in order to determine concept feasibility. If an 
acceptable safety level can be demonstrated in these 
situations, then operations may be conducted with lower 
separation minimums. 

An experiment was conducted in NASA Langley’s Air 
Traffic Operations Lab to address issues associated with 
resolving near-term conflicts and the potential use of lower 
separation minimums. Sixteen commercial airline pilots 
flew a total of 32 traffic scenarios that required them to use 
prototype ASAS tools to resolve close range “pop-up” 
conflicts. Required separation standards were set at either 
3 or 5 NM lateral spacing, with 1000 ft vertical separation 
being used for both cases. Reducing the lateral separation 
from 5 to 3 NM did not appear to increase operational risk, 
as indicated by the proximity to the intruder aircraft. Pilots 
performed better when they followed tactical guidance 
cues provided by ASAS than when they didn't follow the 
guidance. As air-air separation concepts are evolved, 
further studies will consider integration issues between 
ASAS and existing Airborne Collision Avoidance Systems 
(ACAS). 


Introduction 

NASA is investigating a radically new concept of 
operations for the future National Airspace System (NAS). 
This concept, known as Distributed Air/Ground Traffic 
Management (DAG-TM), enables pilots flying 
appropriately equipped “autonomous” aircraft to have 
more flexibility in choosing optimal flight trajectories 
while simultaneously resolving conflicts with other 
aircraft, special use airspace, and hazardous weather [1], 
Air Traffic Service Providers still provide constraints in 
this concept when needed to meet local traffic flow 
management needs and also provide traffic separation 
services to those aircraft unable to participate in 
autonomous operations. A critical component to the 
feasibility of DAG-TM operations is the effectiveness of 
an Airborne Separation Assurance System (ASAS) aboard 
the autonomous aircraft. 

Separation assurance systems would normally enable 
pilots operating in a DAG-TM environment to respond to 
traffic conflict situations well before the conflicting 
aircraft poses a hazard to safe flight. Previous studies have 
shown that pilots are able to use prototype ASAS tools 
effectively to resolve longer-term conflicts [2-3]. 

One can envision situations that may arise, however, 
that do not allow this normal opportunity for strategic 
decision-making. Various non-normal events could 
require the autonomous aircraft pilot to use the ASAS 
system to resolve a near-term conflict. Examples may 
include an aircraft forced to descend rapidly due to a loss 
of cabin pressure or an improper maneuver by an aircraft 
responding to another conflict. Demonstration of a pilot's 
ability to effectively use ASAS to resolve near-term 
conflicts and regain lost separation is needed in order to 
assess concept feasibility. 



These types of non-normal events will require the 
ASAS to provide effective alerts and resolutions prior to 
the time that an Airborne Collision Avoidance System 
(ACAS) would give a Resolution Advisory (RA). When 
an RA is issued, a pilot must take immediate action in 
order to avoid a potential near miss. The Traffic Alert and 
Collision Avoidance System (TCAS) II currently functions 
as an ACAS aboard commercial aircraft. Depending on 
the own aircraft's altitude, TCAS only issues RA's 15-35 
seconds prior to the Closest Point of Approach (CPA) [4], 
Prior to an RA, DAG-TM pilots operating autonomous 
aircraft must rely solely on ASAS for resolution guidance. 

An additional area of DAG-TM concept feasibility 
relates to a potential reduction in separation standards. 
Lower separation standards are likely needed in order to 
improve NAS efficiency and capacity [5-6]. Current 
separation minimums are based in large part on the 
capabilities of older radar systems [7]. Safety assessments 
are needed to determine the feasibility of reduced 
separation minimums. They will give strong consideration 
to surveillance system performance, including accuracy, 
integrity, and availability [8-10]. Candidate surveillance 
systems include Automatic Dependent Surveillance- 
Broadcast (ADS-B) and multi-lateration systems. 

Considering studies done for Reduced Vertical Separation 
Minimums (RVSM) operations, it is likely that flight 
technical errors will also be considered [11-12]. 

In addition to a thorough evaluation of surveillance 
system performance, a potential decision to lower the 
separation standards should also take operational 
considerations into account. An ASAS Safety Assessment 
study identified improper maneuvering in response to a 
conflict (due to ambiguous or improper resolution 
commands or a pilot’s failure to comply with the 
resolution) as a potential safety risk [9]. If near-term 
conflicts with lower separation minimums were 
determined to be more challenging for pilots, the severity 
of these risks could be even greater. 


An experiment was conducted in NASA Langley’s Air 
Traffic Operations Lab to address issues related to ASAS 
use for near-term conflicts with a potential reduction in 
separation minimums. The experiment had the following 
two primary objectives: 

• Evaluate the effectiveness of prototype ASAS 
tools in enabling the pilot to safely resolve 
near-term conflicts. 

• Compare the effect of 3 and 5 NM lateral 
separation standards (with 1000 ft vertical 
separation) on a pilot’s ability to safely 
resolve near-term traffic conflicts and regain 
lost separation. 

The experiment was part of a series of ongoing air-air 
separation assurance studies, including previous work 
presented at the 4 th USA/Europe Air Traffic Management 
R&D Seminar [3]. 

Experimental Approach 

Air Traffic Operations Lab 

The Air Traffic Operations Lab at NASA Langley is a 
medium fidelity PC workstation-based facility, enabling 
simultaneous operation by up to 8 subject pilots. Each 
pilot station consists of a transport aircraft model and flight 
deck displays designed to replicate the MD-11. Figure 1 
shows the display and flight control suite made available to 
the pilots during the experiment. Traffic information was 
superimposed on the navigation display and pilots were 
provided with a Navigation Display Control Panel (NDCP) 
to adjust the display’s traffic specific features. The NDCP 
was located to the left of the Glareshield Control Panel. 
All control of the simulation aircraft was done through the 
Glareshield Control Panel and Multifunction Control 
Display Unit (MCDU) associated with the Flight 
Management System (FMS). No manual flight control 
was available. 
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Figure 1. Pilot Displays and Control Panels 


Subjects 

Sixteen commercial airline pilots with experience in 
Airbus glass cockpit or MD-1 1 aircraft participated in the 
experiment. All were active pilots or had retired within the 
previous year. Their ages ranged from 32 to 57 and flight 
experience ranged from 4,200 to 23,000 hours. 

Design 

The experiment used a single -factor within-subjects 
design. This factor was the required lateral separation 
from other aircraft and was set at either 3 or 5 NM. The 
required vertical separation was 1000 ft for both cases. 

Pilots flew one scenario for each lateral separation 
condition and each scenario had one designed near-term 
conflict. Although other background aircraft were present, 
the data analysis only considered the designed conflict. 
Scenarios for the lateral separation condition were 
counterbalanced for order and were included as part of a 
larger study that consisted of nine scenarios flown per 
pilot. Several of the other seven scenarios were run in 
between those for this experiment, in order to help prevent 
the pilot from anticipating the designed conflict. This 
paper only discusses results from the two near-term 
conflict scenarios. 


Scenario Set-up and Pilot Tasks 

All pilots were flying autonomous aircraft in a DAG- 
TM en-route environment and were asked to maintain 
standard separation from other air traffic and from 
restricted airspace. Pilots were allowed to maneuver freely 
without contacting a controller. This experiment focused 
only on air-air separation assurance involving 
“autonomous” aircraft and the simulation did not 
incorporate a ground component. Future studies are 
planned that will look at integrated air-ground applications 
involving aircraft of mixed equipage. 

In addition to maintaining traffic separation, pilots 
were given a downstream waypoint to cross at a Required 
Time of Arrival (RTA). En route to the RTA, the aircraft’s 
programmed flight path went through a 65 NM wide 
corridor with restricted airspace areas on each side. Pilots 
were asked to continue to the RTA waypoint when able 
after resolving a conflict. The designed conflict occurred 
about 15 minutes into a 25 -minute scenario. 

In order to simulate an unexpected near-term conflict, 
the designated intruder aircraft was hidden from the 
subject pilot until just before the predicted loss of 
separation. At this point, a “pseudo pilot” would turn the 
intruder toward the subject and turn on the intruder’s ADS- 
B broadcast. It appeared about 6 NM away from and at 
co-altitude with ownship. An alert occurred shortly 
thereafter. 
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The initial conflict geometry (location, approach angle, 
and time to closest approach) was designed to be the same 
for both 3 NM and 5 NM separation zone cases. Pseudo 
pilots were used to ensure the conflict occurred, even if the 
subject pilot had deviated from the original programmed 
flight plan. Use of the same initial conflict conditions for 
all scenarios allowed a comparison of risk incurred with 
the 3 NM and 5 NM separation zones. 

For the 5 NM separation zone condition, separation 
loss occurred earlier, but further away from the intruder 
when compared to a 3 NM zone. These differences would 
likely highlight any variation in pilot performance due to 
an aircraft being inside or outside the separation zone. As 
discussed below, alert symbology changed slightly during 
a loss of separation. 

Because the intruder in the conflict scenarios always 
appeared close to the ownship, a highly aggressive 
maneuver would have been required to avoid a separation 
violation. In order to evaluate a pilot’s ability to regain 
separation, avoiding a separation loss was designed to be 
nearly impossible for the scenarios used in this experiment. 

Depending on selected map range, the subject pilot 
could see additional scripted aircraft on the traffic display 
while en-route to the RTA waypoint. An automatic traffic 
filter showed other aircraft determined to be operationally 
significant, based on proximity, current trajectory, and 
intent. 

After each scenario, pilots were asked to complete a 
questionnaire asking them to rate the effectiveness of the 
traffic alerts and resolution guidance in resolving the 
conflicts. They were also asked to assess the safety risk 
posed by the pop-up conflicts. 

Displays and Alerts 

The prototype ASAS system provided conflict 
detection, prevention, and resolution based on both state 
(position, ground speed, and ground track) and FMS-based 
intent information. ASAS tools were part of larger 
decision aiding system, known as the Autonomous 
Operations Planner [13]. TCAS was not available for the 
experiment. 

Traffic alerts were based on outer and inner zones 
surrounding the aircraft, referred to as the separation and 
collision zones, respectively (see Figure 2). This design 


was based on the concept described by RTCA’s Airborne 
Conflict Management (ACM) committee [10]. The 
separation zone represents the minimum legal separation 
around an aircraft. It had a 3 or 5 NM radius, depending 
on the scenario. The vertical dimension was 1000 ft for 
both cases. A conflict was defined as a predicted loss of 
separation and a separation violation occurred if an aircraft 
penetrated another aircraft’s separation zone. The 
collision zone’s radius and height above/below were 900 ft 
and 300 ft, respectively. Penetration of the collision zone 
was considered to be a near miss. 

Separation Zone 



Not to Scale 

Figure 2. Separation and Collision Zones 

Figure 3 shows a notional timeline for ASAS and 
ACAS as applied to this experiment. After a conflict was 
detected but prior to separation loss, ASAS provided 
information to assist the pilot in maintaining adequate 
separation. Strategic and/or tactical separation assurance 
decision support tools were available, depending on the 
time to conflict. The strategic system was integrated with 
the FMS to provide an efficient conflict resolution and 
return to the programmed flight plan. Due to the short 
timeframe of the planned conflicts, pilots were not able to 
use the strategic system for this study. Tactical conflict 
resolutions used only state information and provided 
recommended changes in heading and/or vertical speed. 
The transition from the strategic to the tactical mode of 
operation occurred between 2 and 5 minutes prior to the 
conflict, which was outside the timeframe of the pop-up 
traffic. Further details of these systems are provided by 
Wing etal. [3,14], 
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Collision Zone 
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Time to Conflict (min) 
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TCAS Not Available for Experiment 

Figure 3. Notional ASAS/ACAS Timeline 

If separation was lost, tactical resolutions continued to 
provide guidance to the pilot on ways to regain the 
minimum separation. In these cases, pilots were instructed 
in a pre-flight briefing to follow the tactical guidance. If 
applicable, an alert was issued one minute prior to 
predicted collision zone entry (considered on the timeline 
to be close to the CPA). Note that the loss of separation 
could shift left or right and occur before or after a collision 
zone alert, depending on the relative velocity between the 
two aircraft. For this experiment, a collision zone alert, if 
issued, normally occurred prior to separation loss. 
Issuance of this alert did not affect the tactical resolutions. 


to the CPA. This timeframe may be considered to be near 
a transition between separation assurance and collision 
avoidance. Because TCAS was not incorporated, no 
explicit effort was made to consider integration of ACAS 
RA’s and AS AS resolutions in this simulation. 

Figure 4 shows Primary Flight and Navigation/Traffic 
Displays as typically seen by the pilot shortly after the 
intruder aircraft appeared and just prior to separation loss. 
The intruder was well inside the conflict detection 
threshold and therefore the conflict alert was generated 
almost immediately. This alert was shown as an amber 
band along the ownship’s flight path where the separation 
loss was predicted to occur. The intruder was also color- 
coded amber to represent its threat level. 

When a state -based conflict was detected, pilots were 
presented with a tactical resolution, shown as green bugs 
on the heading and vertical speed indicators. The bugs 
showed the recommended change in either heading or 
vertical speed needed to resolve the conflict. This 
resolution guidance allowed the pilot to resolve the 
conflict by following either the lateral or vertical guidance 
alone. The resolution algorithm used a modified voltage 
potential method, originally developed by Eby [15] and 
refined by the NLR [16]. This algorithm acts to increase 
the projected separation between the two aircraft at CPA. 

Conflict prevention bands were placed on the heading 
and vertical speed indicators to show headings and vertical 
speeds that, if flown, would cause a conflict with another 
aircraft. These bands were predicted using current state 
information for up to a five -minute time horizon. The 
heading and vertical speed bands assumed a constant 
ground speed maneuver in either the horizontal or vertical 
plane, respectively. A change in speed or a combined 
lateral/vertical maneuver would change the conflict 
prevention bands. Once inside another aircraft's 
separation zone, conflict prevention bands were removed. 
After separation loss, the bands no longer provided useful 
information because they would span the entire heading 
and vertical speed range. 


The experiment focused on maneuvers performed by 
pilots just prior to when TCAS would have provided an 
RA. As discussed earlier, TCAS issues RA's between 15 
and 35 seconds prior to CPA. Conflict alerts during the 
experiment occurred mainly between 60-70 seconds prior 
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Figure 4. Primary Flight (left) and Navigation (right) Displays 


Results 

Performance Metrics 


For this example, s (using a 5 NM ellipsoid) equals 
2.25 and 0.002 for the vertical and lateral cases, 
respectively. These values effectively capture the 
difference in risk level. 


A single parameter that combines the relative lateral 
and vertical distances between the ownship and intruder 
aircraft was used as a measure of threat severity. This 
parameter, referred to as epsilon (s), is discussed by den 
Braven [17]. It represents the ellipse distance of an 
ellipsoid enclosed within the cylindrical separation zone. 

Figure 5 shows a cross-section of this geometry for an 
ellipsoid corresponding to a 5 NM separation standard. 
The lengths of the ellipsoid's major and minor axes are the 
separation zone’s diameter and total height, respectively. 
When 8=1, an aircraft is on the surface of the ellipsoid 
centered on another aircraft. Once inside the separation 
zone, threat severity increases as e approaches zero. 
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Epsilon was considered to be a better risk metric than 
aircraft separation at CPA. The latter does not account for 
the separation zone’s different lateral and vertical 
dimensions. For example, a 1500 ft separation at CPA 
could represent two significantly different risk situations. 
In one case, an aircraft flying 1500 ft directly over another 
would be safely outside the separation zone. If that same 
separation were entirely in the lateral plane, a near miss of 
!4 NM would have occurred. 


Figure 5. Ellipsoid and Cylindrical Separation Zone 

In order to compare results from the 3 and 5 NM 
scenarios, all results using e were calculated relative to a 5 
NM ellipsoid. With this convention, the same value of e 
corresponded to the same threat level for both separation 
standards. For cases where the tested separation zone was 
3 NM, e<l did not necessarily imply a separation violation 
(Figure 5, Region A). Conversely, a separation violation 
may still have occurred when e>l, for cases where the 
aircraft was outside the ellipsoid, but within the separation 
zone cylinder (Figure 5, Region B). 
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Two performance metrics were considered for the 
experiment: 


• Threat Proximity (E IIlin ): actual minimum e 
between the two aircraft. 

• Risk Mitigation (s dlf f): difference between the 
predicted minimum e at the time the alert was 
issued (based on current state information for 
both aircraft) and s^n- 

Threat Proximity 

Figure 6 shows for the 3 and 5 NM separation 
cases, broken down by whether the pilot complied or did 
not comply with the tactical resolution. The pilot was said 
to comply if his initial maneuver (either lateral or vertical) 
was in the same direction as the corresponding tactical 
resolution. Using this definition, the pilot complied 9/15 
(60%) times for the 3 NM separation zone and 13/16 
(81%) times for the 5 NM zone. Note that one 3 NM 
separation scenario was lost due to an earlier pilot 
maneuver that prevented the planned conflict from taking 
place. 
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Figure 6. Minimum Epsilon by Separation Zone Size 
and Tactical Resolution Compliance 


The lateral separation zone distance did not appear to 
affect threat proximity. No significant differences were 
found for E rlllII between the 3 and 5 NM separation cases 
when combined over pilot compliance. 


complied (M = .497, SD = .310, n = 22) than for those who 
did not comply ( M = .266, SD = .193, n = 9). Observed 
power ( 1 - P ) was .515. 

Across all runs, the lowest observed value of e^ 
occurred for a 5 NM separation zone condition where the 
pilot did not comply with the tactical resolution. For this 
individual scenario, £ lnln and the lateral and vertical aircraft 
separation at £ mm were 0.015, 0.35 NM, and 100 ft, 
respectively. In this case, s^n occurred at the same time as 
the CPA, however, E Illln generally occurred prior to the 
CPA for other scenarios in this experiment. As discussed 
earlier, £ lnln was considered to better represent operational 
risk than aircraft separation at CPA. 

Risk Mitigation Due to Maneuver 

Whereas is a measure of maximum threat severity, 
Ed® can be used to assess a pilot's ability to improve the 
conflict situation over that predicted when the alert was 
first issued. Epsilon^ is a more suitable metric for 
comparing pilot performance because it accounts for any 
differences in predicted £ lmll that may have occurred due to 
different initial conflict geometry. Although the 
experiment was designed to generate the same initial 
conditions (location, approach angle, and time to closest 
point of approach) across all scenarios, prior maneuvers by 
the subject pilot caused differences in a few cases. On 
average, however, these differences were minimal. 
Predicted £ mln at the time of the alert ranged from 0.01 for 
the 3 NM/did-not-comply scenarios to 0.07 for the 5 
NM/comply cases, with an overall mean across all 
scenarios of 0.04. 

Figure 7 shows £ d iff as a function of separation zone 
size and resolution compliance. A linear regression of 
tactical guidance compliance (combined across separation 
zone conditions) onto £ diff showed that compliance could 
marginally predict £ diff (F( 1 , 29) = 3.118,/? = 0.088, /T’. ld| = 
.066) and that s di ff was marginally larger for those who 
complied with guidance ( M = .446, SD = .297 , n = 22) than 
those who did not comply ( M = .256, SD = .192, n = 9). 
Observed power ( 1 - P ) was .400. This result was not 
significant at the a = .05 level used for the present study. 


A linear regression of tactical guidance compliance 
(combined across separation zone conditions) onto E lnlll 
was performed. The regression showed that compliance 
could significantly predict £ mln (F( 1 , 29) = 4.264, p < .05, 
R 2 a dj = -098) and that £ lmn was larger for those who 


The lateral separation zone distance did not appear to 
affect the pilot's risk mitigation. No significant 
differences were found for £ dlf f between the 3 and 5 NM 
separation cases when combined over pilot compliance. 
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Figure 7. Differences between Actual Minimum Epsilon 
and Minimum Epsilon Predicted when Alert Issued, by 
Separation Zone Size and Tactical Resolution 
Compliance 


Pilot Questionnaires 

Post-scenario questionnaire results related to the 
decision support tools and the pilots' perceived operational 
safety are shown in Table 1. Each question had a 1 to 7 
rating scale, ranging from 1 (least favorable) to 7 (most 
favorable). The scale description as applied to each 
question is given in the 2 nd column of Table 1. 

Table 1. Pilot Questionnaire Results 


Results suggest that pilots had a weak acceptance of 
the conflict alerts and resolutions. These results may be 
partially due to pilots only having an opportunity to use the 
ASAS tools during high-risk short-term conflicts. 
Potential design changes to the ASAS tools currently 
under consideration and discussed below may also 
improve these ratings for future studies. It is not surprising 
that pilots felt neutral about the level of safety 
corresponding to a pop-up conflict. These conflicts were 
designed to pose a safety hazard associated with a non- 
normal event. 

The DAG-TM concept incorporates several safety 
measures to reduce the likelihood of the type of scenario 
evaluated in this experiment [1], Before maneuvering, 
pilots of autonomous aircraft must ensure that they do not 
create near-term conflicts. Refined ASAS tools relying on 
highly dependable surveillance systems should provide 
these aircraft with adequate warning of upcoming conflicts 
in all nominal cases. During normal operations, pilots of 
both aircraft will have the necessary tools to resolve 
conflicts. New decision support tools are also expected to 
enable air traffic service providers to provide separation 
services for aircraft not equipped for autonomous 
operations. 


Question 

Rating Scale 

Overall 

Mean 

How intuitive was 
the conflict alerting 
system? 

1 : not at all intuitive 
— » 7 : very intuitive 

5.0 

How acceptable 
were the tactical 
resolutions? 

1 : not at all 
acceptable — > 7: 
completely 
acceptable 

4.3 

What was the level 
of safety for this 
scenario? 

1 : completely unsafe 
— > 7: completely 
safe 

4.0 

How did the 
conflict manage- 
ment tools affect 
the risk level? 

1 : greatly increased 
risk — > 7: greatly 
decreased risk 

4.8 
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Discussion 

Reducing the lateral separation from 5 to 3 NM did not 
appear to increase operational risk for the near-term 
conflicts studied in this experiment. Additional studies 
could consider whether any reduction in alert and 
subsequent maneuver times would lead to lower risk in 
cases where a conflict situation exists prior to the alerting 
time horizon. Because a 3 NM separation zone represents 
a smaller target, trajectory uncertainties may cause a 
shorter alert time when compared to a 5 NM zone. A 
smaller separation zone may also reduce the number of 
alerted conflicts that evolve into safety critical situations. 
Due to the consistent alert time associated with the pop-up 
conflicts in this experiment, no such changes could be 
observed. As discussed earlier, extensive analysis is also 
needed to determine whether surveillance system 
performance can support lower separation minimums. 

Pilots were able to reduce threat severity and mitigate 
risk in response to pop-up conflicts more effectively when 
they followed the tactical guidance from ASAS. In order 
to further improve upon those results, several design 
enhancements could be considered for the conflict 
resolution system. These changes could tap into extensive 
development and evaluation of TCAS. 

As the time to closest point of approach decreases, it 
becomes more important for the ASAS resolutions to be 
compatible with RA's that are eventually provided by 
ACAS [8]. TCAS design goals that could be considered 
for ASAS enhancement include: attempt to avoid crossing 
the intruder’s altitude (especially for very close 
encounters), provide a resolution that does not require the 
ownship to change direction, allow time for each aircraft to 
initiate a maneuver prior to reversing the RA, and avoid 
reversing the RA unless needed to ensure safety [4, 18-19]. 
These design goals are likely extensible to both lateral and 
vertical maneuvers. For example, previous studies have 
shown that pilots are less likely to turn toward the intruder 
as they get closer to the conflict [20]. 

In addition to the desirable goal of more compatible 
ASAS and ACAS systems [6, 10, 21], enhancements to the 
ASAS tools that consider pilot maneuver preferences when 
possible would also likely increase the compliance rate and 
response time [22]. Previous studies have shown that 
pilots are more likely to comply promptly and correctly 
with resolution commands when the guidance supports 
their maneuver tendencies [22-23] . 


Conclusions 

For near-term conflicts, a pilot’s ability to reduce threat 
proximity and mitigate risk appears to depend more on 
compliance with the ASAS resolution guidance than on the 
size of the lateral separation zone. Compliance with the 
tactical resolutions led to improvement in threat proximity 
and risk mitigation. These positive effects may be further 
increased by design enhancements to the ASAS conflict 
resolution system. 

Before air-air separation assurance or other applications 
with lower separation minimums are implemented, a 
thorough Operational Safety Assessment will need to be 
performed. This process has been described by Zeitlin [8]. 
Included in this process is an Operational Flazard Analysis 
where potential hazards, the operational effects of those 
hazards, and mitigation strategies are assessed. Evaluation 
of pilot performance using ASAS tools will likely be an 
important part of this process. 
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